System and method employing individual user content-based data and user collaborative feedback data to evaluate the content of an information entity in a large information communication network

ABSTRACT

An information entity rating system includes a content subsystem having a structured data sub-subsystem and an unstructured data sub-subsystem. The content subsystem receives content-based profile data for an information entity and separately processes structured and unstructured data to combine content-based profile data for an individual system user with the content-based profile data for the information entity to determine computed rating functions indicating structured and unstructured content-based value of the information entity to the user. A collaboration subsystem receives collaborative input data for the information entity and for processes the collaborative input data to determine at least one computed collaborative rating function indicating a collaboration-based value of the information entity to the user. A correlation subsystem receives data from the content subsystem and from the collaboration subsystem to determine exceptions to the computed rating functions on the basis of comparisons of data included in the content-based and collaboration data and to generate an exception data value function indicating an opposing value to at least one of the content-based and collaboration values. An output system combines the structured content-based. unstructured content-based, and collaboration-based value functions, and the exception data value function in generating an output rating predictor of the informon for consideration by the user.

This application is a continuation of application Ser. No. 08/627,436,filed Apr. 4, 1996, now U.S. Pat. No. 5,867,799.

BACKGROUND OF THE INVENTION

The present invention relates to information processing systems forlarge or massive information networks, such as the internet, and moreparticuarly to such information systems in which an information filterstructure uses collaborative feedback data in determining the value of adocument or other information entity (informon) to a user.

In the operation of the internet, a countless number of informons areavailable for downloading from any of at least thousands of sites forconsideration by a user at the user's location. A user typicaly connectsto a portal or other web site having a search capability, and thereafterenters a particular query, i.e., a request for informons relevant to atopic, a field of interest, etc. Thereafter, the search site typicallyemploys a "spider" scanning system and a content-based filter in asearch engine to search the internet for informons which match thequery. This process is basically a pre-search process in which matchinginformons are found, at the time of initiating the search for the user'squery. by comparing informons in an "informon data base" to the user'squery.

The return list of matching informons can be very extensive according tothe subject of the query and the breadth of the query. More specificqueries typically result in shorter return lists. In some cases, thesearch site may also be structured to find web sites which probably havestored informons matching the entered query.

Collaborative data can be made available to assist in informon ratingwhen a user actually downloads an informon, considers and evaluates it,and returns data to the search site as a representation of the value ofthe considered informon to the user.

In the patent application parent to this divisional application, i.e.,Ser. No. 08/627,436, now U.S. Pat. No. 5,867,799, filed by the presentinventors on Apr. 4, 1996, and hereby incorporated by reference, anadvanced collaborative/content-based information filter system isemployed to provide superior fitering in the process of finding andrating informons which match a user's query. The information filterstructure in this system integrates content-based filtering andcollaborative filtering to determine relevancy of informons receivedfrom various sites in the internet or other network. In operation, anindividual user enters a query and a corresponding "wire" isestablished, i.e., the query is profiled in storage on a content basisand adaptively updated over time, and informons obtained from thenetwork are compared to the profile for relevancy and ranking. Acontinuously operating "spider" scans the network to find informonswhich are received and processed for relevancy to the individual user'swire and for relevancy to wires established by numerous other users.

The integrated filter system compares received informons to theindividual user's query profile data, combined with collaborative data,and ranks, in order of value, informons found to be relevant. The systemmaintains the ranked informons in a stored list from which theindividual user can select any listed informon for consideration.

As the system continues to operate the individual user's wire, thestored relevant informon list typically changes due to factors includinga return of new and more relevant informons, adjustments in the user'squery, feedback evaluations by the user for considered informons, andupdatings in collaborative feedback data. Received informons aresimilarly processed for other users' wires established in theinformation filter system. Thus, the integrated information filtersystem compares network informons to multiple user's queries to findmatching informons for various users' wires over the course of time,whereas conventional search engines initiate a search in response to anindividual user's query and use content-based filtering to compare thequery to accessed network informons to find matching informons during alimited search time period.

The present invention is directed to an informon rating system in whichcontent-based filter profile data and collaborative feedback filter dataare integrated and compared to data representative of an informon beingrated to determine the relevancy and value of the informon to anindividual user. This system is embodied in the multi-level, integratedcollaborative/content-based filter disclosed in the parent application,and it receives informon data, which is passed downwardly through thefilter structure, and collaborative feedback data which is sent from acollaborative feedback data processsing system called a mindpool system.Another copending patent application, entitled MULTI-LEVEL MINDPOOLSYSTEM ESPECIALLY ADAPTED TO PROVIDE COLLABORATIVE FILTER DATA FOR ALARGE-SCALE INFORMATION FILTERING SYSTEM, Serial Number (Atty. docket #LYC2), filed by the current inventors concurrenty herewith, providesfurther discosure and explanation of the mindpool system.

SUMMARY OF THE INVENTION

An information entity rating system comprises a content subsystem forreceiving content-based profile data for an information entity and forcombining content-based profile data for an individual system user withthe content-based profile data for the information entity to determineat least one computed rating function indicating a content-based valueof the information entity to the user. A collaboration subsystemreceives collaborative input data for the information entity andprocesses the collaborative input data to determine at least onecomputed collaborative rating function indicating a collaboration-basedvalue of the information entity to the user. An output subsystemcombines the content-based and collaboration-based value functions togenerate an output rating predictor of the informon for consideration bythe user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an diagrammatic representation of an embodiment of aninformation filtering apparatus according to the present invention.

FIG. 2 is an diagrammatic representation of another embodiment of aninformation filtering apparatus according to the present invention.

FIG. 3 is a flow diagram for an embodiment of an information filteringmethod according to the present invention.

FIG. 4 is a flow diagram for another embodiment of an informationfiltering method according to the present invention.

FIG. 5 is a flow diagram for yet another embodiment of an informationfiltering method according to the present invention.

FIG. 6 is an illustration of a three-component-input model and profilewith associated predictors.

FIG. 7 is an illustration of a mindpool hierarchy.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The invention disclosed herein is embodied in an apparatus and methodfor information filtering in a computer system receiving a data streamfrom a computer network, in which entities of information relevant tothe user, or "informons," are extracted from the data stream usingcontent-based and collaborative filtering. The information filtering isboth interactive and distributed in structure and method. It isinteractive in that communication is substantially bi-directional ateach level of the filter. It is distributed in that all or part of theinformation filter can include a purely hierarchical(up-and-down/parent-child) structure or method, a purely parallel(peer-to-peer) structure or method, or a combination of hierachical andparallel structures and method.

As used herein, the term "informon" comprehends an information entity ofpotential or actual interest to a particular user. In general, informonscan be heterogenous in nature and can be all or part of a textual, avisual, or an audio entity. Also, informons can be composed of acombination of the aforementioned entities, thereby being a multimediaentity. Furthermore, an informon can be an entity of patterned data,such as a data file containing a digital representation of signals andcan be a combination of any of the previously-mentioned entities.Although some of the data in a data stream, including informons, may beincluded in an informon, not all data is relevant to a user, and is notwithin the definition of an informon. By analogy, an informon may beconsidered to be a "signal," and the total data stream may be consideredto be "signal+noise." Therefore, an information filtering apparatus isanalogous to other types of signal filters in that it is designed toseparate the "signal" from the "noise."

Also as used herein, the term "user" is an individual in communicationwith the network. Because an individual user can be interested inmultiple categories of information, the user can be considered to bemultiple clients each having a unique profile, or set of attributes.Each member client profile, then, is representative of a particulargroup of user preferences. Collectively, the member client profilesassociated with each user is the user profile. The present invention canapply the learned knowledge of one of a user's member clients to othersof the user's member clients, so that the importance of the learnedknowledge, e.g., the user's preference for a particular author in oneinterest area as represented by the member client, can increase theimportance of that particular factor, A's authorship, for others of theuser's member clients. Each of the clients of one user can be associatedwith the individual clients of other users insofar as the profiles ofthe respective clients have similar attributes. A "community" is a groupof clients, called member clients, that have similar member clientprofiles, i.e., that share a subset of attributes or interests. Ingeneral, the subset of shared attributes forms the community profile fora given community and is representative of the community norms, orcommon client attributes.

The "relevance" of a particular informon broadly describes how well itsatisfies the user's information need. The more relevant an informon isto a user, the higher the "signal" content. The less relevant theinformon, the higher the "noise" content. Clearly, the notion of what isrelevant to a particular user can vary over time and with context, andthe user can find the relevance of a particular informon limited to onlya few of the user's potentially vast interest areas. Because a user'sinterests typically change slowly, relative to the data stream, it ispreferred to use adaptive procedures to track the user's currentinterests and follow them over time. Provision, too, is preferred to bemade for sudden changes in interest, e.g., taking up antiquarian swordcollecting and discontinuing stamp collecting, so that the method andapparatus track the evolution of "relevance" to a user and thecommunities of which the user is a member. In general, informationfiltering is the process of selecting the information that a userswishes to see, i.e., informons, from a large amount of data.Content-based filtering is a process of filtering by extracting featuresfrom the informon, e.g., the text of a document, to determine theinformon's relevance. Collaborative filtering, on the other hand, is theprocess of filtering informons, e.g., documents, by determining whatinformons other users with similar interests or needs found to berelevant.

The invention employs adaptive content-based filters and adaptivecollaborative filters, which respectively include, and respond to, anadaptive content profile and an adaptive collaboration profile. As usedherein, the term "content-based filter" means a filter in which contentdata, such as key words, is used in performing the filtering process. Ina collaborative filter, other user data is used in performing thefiltering process. A collaborative filter is also sometimes referred toas a "content" filter since it ultimately performs the task of findingan object or document having content relevant to the content desired bya user. If there are some instances herein where the term "contentfilter" is used as distinguished from a collaborative filter, it isintended that the term "content filter" mean "content-based filter."Theadaptive filters each are preferred to include at least a portion of acommunity filter for each community serviced by the apparatus, and aportion of a member client filter for each member client of the servicedcommunities. For this reason, the adaptive filtering is distributed inthat each of the community filters perform adaptive collaborativefiltering and adaptive content filtering, even if on different levels,and even if many filters exist on a given level. The integratedfiltering permits an individual user to be a unique member client ofmultiple communities, with each community including multiple memberclients sharing similar interests. The adaptive features permit theinterests of member clients and entire communities to change graduallyover time. Also a member client has the ability to indicate a suddenchange in preference, e.g., the member client remains a collector but isno longer interested in coin collecting.

The filter structure also implements adaptive credibility filtering,providing member clients with a measure of informon credibility, asjudged by other member clients in the community. For example, a newmember client in a first community, having no credibility, can inject aninformon into the data flow, thereby providing other member clients inother communities with the proposed informon, based on the respectivecommunity profile and member client profiles. If the other memberclients believe the content of the informon to be credible, the adaptivecredibility profile will reflect a growing credibility. Conversely,feedback profiles from informon recipients that indicate a lack ofcredibility cause the adaptive credibility profile, for the informonauthor, to reflect untrustworthiness. However, the growth anddeclination of credibility are not "purely democratic," in the sensethat one's credibility is susceptible to the bias of others'perceptions, so the growth or declination of one's credibility isgenerally proportional to how the credibility of the new member clientis viewed by other member clients.

Member clients can put their respective reputations "on the line," andengage in spirited discussions which can be refereed by other interestedmember clients. The credibility profile further can be partitioned topermit separate credibility sub-profiles for the credibility of thecontent of the informon, the author, the author's community, thereviewers, and the like, and can be fed back to discussion participants,reviewers, and observers to monitor the responses of others to thedebate. The adaptive credibility profiles for those member clients withtop credibility ratings in their communities may be used to establishthose member clients as "experts" in their respective communities.

With this functionality, additional features can be implemented,including, for example, "instant polling" on a matter of political orconsumer interest. In conjunction with both content and collaborativefiltering, credibility filtering, and the resulting adaptive credibilityprofiles, also may be used to produce other features, such as on-lineconsultation and recommendation services. Although the "experts" in thecommunities most closely related to the topic can be afforded specialstatus as such, member clients from other communities also canparticipate in the consultation or recommendation process.

In one embodiment of the consultation service, credibility filtering canbe augmented to include consultation filtering. With this feature, amember client can transmit an informon to the network with a request forguidance on an issue, for example, caring for a sick tropical fish.Other member clients can respond to the requester with informons relatedto the topic, e.g., suggestions for water temperature and antibiotics.The informons of the responders can include their respective credibilityprofiles, community membership, and professional or avocationalaffiliations. The requester can provide feedback to each of theresponders, including a rating of the credibility of the responder onthe particular topic. Additionally, the responders can accrue qualitypoints, value tokens, or "info bucks," as apportioned by the requester,in return for useful guidance.

Similarly, one embodiment of an on-line recommendation service usesrecommendation filtering and adaptive recommendation profiles to givemember clients recommendations on matters as diverse as local automechanics and world-class medieval armor refurbishers. In thisembodiment, the requester can transmit the informon to the networkbearing the request for recommendation. Other member clients can respondto the requester with informons having specific recommendations ordisrecommendations, advice, etc. As with the consultation service, theinformons of the responders can be augmented to include their respectivecredibility profiles, community membership, and professional oravocational affiliations. A rating of each recommendation provided by aresponder, relative to other responders' recommendations, also can besupplied. The requester can provide feedback to each of the responders,including a rating of the credibility of the responder on the particulartopic, or the quality of the recommendation. As before, the responderscan accrue quality points, value tokens, or "info bucks," as apportionedby the requester, in return for the useful recommendation.

Furthermore, certain embodiments are preferred to be self-optimizing inthat some or all of the adaptive filters used in the system dynamicallyseek optimal values for the function intended by the filter, e.g.,content analysis, collaboration, credibility, reliability, etc.

The filter structure herein is capable of identifying the preferences ofindividual member clients and communities, providing direct andinferential consumer preference information, and tracking shifts in thepreferences whether the shifts be gradual or sudden. The consumerpreference information can be used to target particular consumerpreference groups, or cohorts, and provide members of the cohort withtargeted informons relevant to their consumer preferences. Thisinformation also may be used to follow demographical shifts so thatactivities relying on accurate demographical data, such as retailmarketing, can use the consumer preference information to anticipateevolving consumer needs in a timely manner.

To provide a basis for adaptation, it is preferred that each rawinformon be processed into a standardized vector, which may be on theorder of 20,000 to 100,000 tokens long. The learning and optimizationmethods that ultimately are chosen are preferred to be substantiallyrobust to the problems which can be presented by such high-dimensionalinput spaces. Dimensionality reduction using methods such as thesingular value decomposition (SVD), or auto-encoding neural networksattempt to reduce the size of the space while initially retaining theinformation contained in the original representation. However, the SVDcan lose information during the transformation and may give inferiorresults. Two adaptation/learning methods that are presently preferredinclude the TF-IDF technique and the MDL technique.

FIG. 1 illustrates one embodiment of an information filtering apparatus1 according to the invention herein. In general, a data stream isconveyed through network 3, which can be a global internetwork. Askilled artisan would recognize that apparatus 1 can be used with othertypes of networks, including, for example, an enterprise-wide network,or "intranet." Using network 3, User #1 (5) can communicate with otherusers, for example, User #2 (7) and User #3 (9), and also withdistributed network resources such as resource #1 (11) and resource #2(13).

Apparatus 1 is preferred to be part of computer system 16, although User#1 (5) is not required to be the sole user of computer system 16. In onepresent embodiment, it is preferred that computer system 16 havinginformation filter apparatus 1 therein filters information for aplurality of users. One application for apparatus 1, for example, couldbe that user 5 and similar users may be subscribers to a commercialinformation filtering service, which can be provided by the owner ofcomputer system 16.

Extraction means 17 can be coupled with, and receives data stream 15from, network 3. Extraction means 17 can identify and extract rawinformons 19 from data stream 15. Each of the raw informons 19 has aninformation content. Extraction means 17 uses an adaptive contentfilter, and at least part of the adaptive content profile, to analyzethe data stream for the presence of raw informons. Raw informons arethose data entities whose content identifies them as being "in theballpark," or of potential interest to a community coupled toapparatus 1. Extraction means 17 can remove duplicate informons, even ifthe informons arrive from different sources, so that user resources arenot wasted by handling and viewing repetitive and cumulativeinformation. Extraction means 17 also can use at least part of acommunity profile and a user profile for User #1 (5) to determinewhether the informon content is relevant to the community of which User#1 is a part.

Filter means 21 adaptively filters raw informons 19 and producesproposed informons 23 which are conveyed to User #1 (5) by communicationmeans 25. A proposed informon is a selected raw informon that, basedupon the respective member client and community profiles, is predictedto be of particular interest to a member client of User 5. Filter means21 can include a plurality of community filters 27a,b and a plurality ofmember client filters 28a-e, each respectively having community andmember client profiles. When raw informons 19 are filtered by filtermeans 21, those informons that are predicted to be suitable for aparticular member client of a particular community, e.g., User #1 (5),responsive to the respective community and member client profiles, areconveyed thereto. Where such is desired, filter means 21 also caninclude a credibility filter which enables means 21 to performcredibility filtering of raw informons 19 according to a credibilityprofile.

It is preferred that the adaptive filtering performed within filtermeans 21 by the plurality of filters 27a,b, 28a-e, and 35, use aself-optimizing adaptive filtering so that each of the parametersprocessed by filters 27a,b, 28a-e, and 35, is driven continually torespective values corresponding to a minimal error for each individualparameter. Self-optimization encourages a dynamic, marketplace-likeoperation of the system, in that those entities having the mostdesirable value, e.g., highest credibility, lowest predicted error,etc., are favored to prevail.

Self-optimization can be effected according to respective preselectedself-optimizing adaptation techniques including, for example, one ormore of a top-key-word-selection adaptation technique, anearest-neighbor adaptation technique, a term-weighting adaptationtechnique, a probabilistic adaptation technique, and a neural networklearning technique. In one present embodiment of the invention, theterm-weighting adaptation technique is preferred to be a TF-IDFtechnique and the probabilistic adaptation technique is preferred to bea MDL technique.

When user 5 receives proposed informon 23 from apparatus 1, user 5 isprovided with multiple feedback queries along with the proposedinformon. By answering, user 5 creates a feedback profile thatcorresponds to feedback response 29. User feedback response 29 can beactive feedback, passive feedback, or a combination. Active feedback caninclude the user's numerical rating for an informon, hints, and indices.Hints can include like or dislike of an author, and informon source andtimeliness. Indices can include credibility, agreement with conyent orauthor, humor, or value. Feedback response 29 provides an actualresponse to proposed informon 23, which is a measure of the relevance ofthe proposed informon to the information need of user 5. Such relevancefeedback attempts to improve the performance for a particular profile bymodifying the profiles, based on feedback response 29.

A predicted response anticipated by adaptive filtering means 21 can becompared to the actual feedback response 29 of user 5 by firstadaptation means 30, which derives a prediction error. First adaptationmeans 30 also can include prediction means 33, which collects a numberof temporally-spaced feedback responses, to update the adaptivecollaboration profile, the adaptive content profile, or both, with anadapted future prediction 34, in order to minimize subsequent predictionerrors by the respective adaptive collaboration filter and adaptivecontent filter.

In one embodiment of the invention herein, it is preferred thatprediction means 33 be a self-optimizing prediction means using apreselected learning technique. Such techniques can include, forexample, one or more of a top-key-word-selection learning technique, anearest-neighbor learning technique, a term-weighting learningtechnique, and a probabilistic learning technique. First adaptationmeans 30 also can include a neural network therein and employ a neuralnetwork learning technique for adaptation and prediction. In one presentembodiment of the invention, the term-weighting learning technique ispreferred to be a TF-IDF technique and the probabilistic learningtechnique is preferred to be a MDL learning technique.

First adaptation means 30 further can include second adaptation means 32for adapting at least one of the adaptive collaboration profiles, theadaptive content profiles, the community profile, and the user profile,responsive to at least one of the other profiles. In this manner, trendsattributable to individual member clients, individual users, andindividual communities in one domain of system 16 can be recognized by,and influence, similar entities in other domains (molding of agent"minds"), contained within system 16 to the extent that the respectiveentities share common attributes.

Apparatus 1 also can include a computer storage means 31 for storing theprofiles, including the adaptive content profile and the adaptivecollaboration profile. Additional trend-tracking information can bestored for later retrieval in storage means 31, or may be conveyed tonetwork 3 for remote analysis, for example, by User #2 (7).

FIG. 2 illustrates another preferred embodiment of information filteringapparatus 50, in computer system 51. Apparatus 50 can include firstprocessor 52, second processors 53a,b, third processors 64a-d, and afourth processor 55, to effect the desired information filtering. Firstprocessor 52 can be coupled to, and receive a data stream 56 from,network 57. First processor 52 can serve as a pre-processor byextracting raw informons 58 from data stream 56 responsive topreprocessing profile 49 and conveying informons 58 to second processors53a,b.

Because of the inconsistencies presented by the nearly-infiniteindividual differences in the modes of conceptualization, expression,and vocabulary among users, even within a community of coincidinginterests, similar notions can be described with vastly different termsand connotations, greatly complicating informon characterization. Modevariations can be even greater between disparate communities,discouraging interaction and knowledge-sharing among communities.Therefore, it is particularly preferred that processor 52 create amode-invariant representation for each raw informon, thus allowing fast,accurate informon characterization and collaborative filtering.Mode-invariant representations tend to facilitate relevant informonselection and distribution within and among communities, therebypromoting knowledge-sharing, thereby benefitting the group ofinterlinked communities, i.e., a society, as well.

First processor 52 also can be used to prevent duplicate informons,e.g., the same information from different sources, from furtherpenetrating, and thus consuming the resources of, the filtering process.Other processors 53,a,b, 54a-d, also may be used to perform theduplicate information elimination function, but additionally may measurethe differences between the existing informon and new informons. Thatdifference between the content of the informon the previous time theuser reviewed it and the content of the informon in its present form isthe "delta" of interest. Processors 53a,b, 54a-d may eliminate theinformon from further processing, or direct the new, altered informon tothe member client, in the event that nature or extent of the changeexceeds a "delta" threshold. In general, from the notion of exceeding apreselected delta threshold, one may infer that the informon has changedto the extent that the change is interesting to the user. The nature ofthis change can be shared among all of a user's member clients. Thisdelta threshold can be preselected by the user, or by the preselectedlearning technique. Such processing, or "delta learning" can beaccomplished by second processors 53a,b, alone or in concert with thirdprocessors 54a-d. Indeed, third processor 54a-d can be the locus fordelta learning, where processors 54a-d adapts a delta learning profilefor each member client of the community, i.e. user, thus anticipatingthose changes in existing informons that the user may find"interesting."

Second processors 53a,b can filter raw informons 58 and extract proposedcommunity informons 59a,b therefrom. Informons 59a,b are those predictedby processors 53a,b to be relevant to the respective communities, inresponse to community profiles 48a,b that are unique to the communities.Although only two second processors 53a,b are shown in FIG. 2, system 51can be scaled to support many more processors, and communities. It ispresently preferred that second processors 53a,b extract communityinformons 59a,b using a two-step process. Where processor 52 hasgenerated mode-invariant concept representations of the raw informons,processor 53a,b can perform concept-based indexing, and then providedetailed community filtering of each informon.

Third processors 54a-d can receive community informons 59a,b fromprocessors 53a,b, and extract proposed member client informons 61a-dtherefrom, responsive to unique member client profiles 62a-d forrespective ones of member clients 63a-d. Each user can be represented bymultiple member clients in multiple communities. For example, each ofusers 64a,b can maintain interests in each of the communities servicedby respective second processors 53a,b, and each receive separate memberclient informons 61b,c and 61a,d, respectively.

Each member client 63a-d provides respective member client feedback65a-d to fourth processor 55, responsive to the proposed member clientinformons 61a-d. Based upon the member client feedback 65a-d, processor55 updates at least one of the preprocessing profile 49, communityprofiles 48a,b and member client profiles 62a-d, . Also, processor 55adapts at least one of the adaptive content profile 68 and the adaptivecollaboration profile 69, responsive to profiles 49, 48a,b, and 62a-d.

Fourth processor 55 can include a plurality of adaptive filters 66a-dfor each of the aforementioned profiles and computer storage therefor.It is preferred that the plurality of adaptive filters 66a-d beself-optimizing adaptive filters. Self-optimization can be effectedaccording to a preselected self-optimizing adaptation techniqueincluding, for example, one or more of a top-key-word-selectionadaptation technique, a nearest-neighbor adaptation technique, aterm-weighting adaptation technique, and a probabilistic adaptationtechnique. Any of the adaptive filter 66a-d, may include a neuralnetwork. In one present embodiment of the invention, the term-weightingadaptation technique is preferred to be a TF-IDF technique and theprobabilistic adaptation technique is preferred to be a MDL technique.

An artisan would recognize that one or more of the processors 52-55could be combined functionally so that the actual number of processorsused in the apparatus 50 could be less than, or greater than, thatillustrated in FIG. 2. For example, in one embodiment of the presentinvention, first processor 52 can be in a single microcomputerworkstation, with processors 53-55 being implemented in additionalrespective microcomputer systems. Suitable microcomputer systems caninclude those based upon the Intel® Pentium-Pro™ microprocessor. Infact, the flexibility of design presented by the invention allows forextensive scalability of apparatus 50, in which the number of users, andthe communities supported may be easily expanded by adding suitableprocessors. As described in the context of FIG. 1, the interrelation ofthe several adaptive profiles and respective filters allow trendsattributable to individual member clients, individual users, andindividual communities in one domain of system 51 to be recognized by,and influence, similar entities in other domains, of system 51 to theextent that the respective entities in the different domains sharecommon attributes.

The above described system operates in accordance with a method 100 forinformation filtering in a computer system, as illustrated in FIG. 3,which includes providing a dynamic informon characterization (step 105)having a plurality of profiles encoded therein, including an adaptivecontent profile and an adaptive collaboration profile; and adaptivelyfiltering the raw informons (step 110) responsive to the dynamicinformon characterization, thereby producing a proposed informon. Themethod continues by presenting the proposed informon to the user (step115) and receiving a feedback profile from the user (step 120),responsive to the proposed informon. Also, the method includes adaptingat least one of the adaptive content profile (step 125) and the adaptivecollaboration profile responsive to the feedback profile; and updatingthe dynamic informon characterization (step 130) responsive thereto.

The adaptive filtering (step 110) in method 100 can be machinedistributed adaptive filtering that includes community filtering(substep 135), using a community profile for each community, and clientfiltering (substep 140), similarly using a member client profile foreach member client of each community. It is preferred that the filteringin substeps 135 and 140 be responsive to the adaptive content profileand the adaptive collaboration profile. Method 100 comprehends servicingmultiple communities and multiple users. In turn, each user may berepresented by multiple member clients, with each client having a uniquemember client profile and being a member of a selected community. It ispreferred that updating the dynamic informon characterization (step 130)further include predicting selected subsequent member client responses(step 150).

Method 100 can also include credibility filtering (step 155) of the rawinformons responsive to an adaptive credibility profile and updating thecredibility profile (step 160) responsive to the user feedback profile.Method 100 further can include creating a consumer profile (step 165)responsive to the user feedback profile. In general, the consumerprofile is representative of predetermined consumer preference criteriarelative to the communities of which the user is a member client.Furthermore, grouping selected ones (step 170) of the users into apreference cohort, responsive to the preselected consumer preferencecriteria, can facilitate providing a targeted informon (step 175), suchas an advertisement, to the preference cohort.

FIG. 4 illustrates yet another preferred method embodiment of of theinvention herein. In general, method 200 includes partitioning (step205) each user into multiple member clients, each having a unique memberclient profile with multiple client attributes and grouping memberclients (step 210) to form a multiple communities with each memberclient in a particular community sharing selected client attributes withother member clients, thereby providing each community with a uniquecommunity profile having common client attributes.

Method 200 continues by predicting a community profile (step 215) foreach community using first prediction criteria, and predicting a memberclient profile (step 220) for a member client in a particular communityusing second prediction criteria. Method 200 also includes a steps ofextracting raw informons (step 225) from the data stream and selectingproposed informons (step 230) from raw informons. The proposed informonsgenerally are correlated with one or more of the common clientattributes of a community, and of the member client attributes of theparticular member client to whom the proposed informon is offered. Afterproviding the proposed informons to the user (step 235), receiving userfeedback (step 240) in response to the proposed informons permits theupdating of the first and second prediction criteria (step 245)responsive to the user feedback.

Method 200 further may include prefiltering the data stream (step 250)using the predicted community profile, with the predicted communityprofile identifying the raw informons in the data stream.

Step 230 of selecting proposed informons can include filtering the rawinformons using an adaptive content filter (step 255) responsive to theinformon content; filtering the raw informons using an adaptivecollaboration filter (step 260) responsive to the common clientattributes for the pertaining community; and filtering the raw informonsusing an adaptive member client filter (step 265) responsive to theunique member client profile.

It is preferred that updating the first and second prediction criteria(step 245) employ a self-optimizing adaptation technique, including, forexample, one or more of a top-key-word-selection adaptation technique, anearest-neighbor adaptation technique, a term-weighting adaptationtechnique, and a probabilistic adaptation technique. It is furtherpreferred that the term-weighting adaptation technique be a TF-IDFtechnique and the probabilistic adaptation technique be a minimumdescription length technique.

The information filtering method shown in FIG. 5 provides rapid,efficient data reduction and routing, or filtering, to the appropriatemember client. The method 300 includes parsing the data stream intotokens (step 301); creating a mode-invariant (MI) profile of theinformon (step 305); selecting the most appropriate communities for eachinformon, based on the MI profile, using concept-based indexing (step310); detailed analysis (step 315) of each informon with regard to itsfit within each community; eliminating poor-fitting informons (step320); detailed filtering of each informon relative to fit for eachmember client (step 325); eliminating poor-fitting informons (step 330);presenting the informon to the member client/user (step 335); andobtaining the member client/user response, including multiple ratingsfor different facets of the user's response to the informon (step 340).

It is preferred that coherent portions of the data stream, i.e.,potential raw informons, be first parsed (step 301) into generalizedwords, called tokens. Tokens include punctuation and other specializedsymbols that may be part of the structure found in the article headers.For example, in addition to typical words such as "seminar" counting astokens, the punctuation mark "$" and the symbol "Newsgroup:comp.ai" arealso tokens. Using noun phrases as tokens also can be useful.

Next a vector of token counts for the document is created. This vectoris the size of the total vocabulary, with zeros for tokens not occurringin the document. Using this type of vector is sometimes called thebag-of-words model. While the bag-of-words model does not capture theorder of the tokens in the document, which may be needed for linguisticor syntactic analysis, it captures most of the information needed forfiltering purposes.

Although, it is common in information retrieval systems to group thetokens together by their common linguistic roots, called stemming, as anext step it is preferred in the present invention that the tokens beleft in their unstemmed form. In this form, the tokens are amenable tobeing classified into mode-invariant concept components.

Creating a mode-invariant profile (step 305), C, includes creating aconceptual representation for each informon, A, that is invariant withrespect to the form-of-expression, e.g., vocabulary andconceptualization. Each community can consist of a "Meta-U-Zine"collection, M, of informons. Based upon profile C, the appropriatecommunities, if any, for each informon in the data stream are selectedby concept-based indexing (step 310) into each M. That is, for eachconcept C that describes A, put A into a queue Q_(M), for each M whichis related to C. It is preferred that there is a list of Ms that isstored for each concept and that can be easily index-searched. Each Athat is determined to be a poor fit for a particular M is eliminatedfrom further processing. Once A has been matched with a particular M, amore complex community profile P_(M) is developed and maintained foreach M (step 315). If A has fallen into Q_(M), then A is analyzed todetermine whether it matches P_(M) strongly enough to be retained or"weeded" out (step 325) at this stage.

Each A for a particular M is sent to each user's personal agent, ormember client U of M, for additional analysis based on the memberclient's profile (step 325). Each A that fits U's interests sufficientlyis selected for U's personal informon, or "U-Zine," collection, Z.Poor-fitting informons are eliminated from placement in Z (step 330).This user-level stage of analysis and selection may be performed on acentralized server site or on the user's computer.

Next, the proposed informons are presented to user U (step 335) forreview. User U reads and rates each selected A found in Z (step 340).The feedback from U can consist of a rating for how "interesting" Ufound A to be, as well as one or more of the following:

Opinion feedback: Did U agree, disagree, or have no opinion regardingthe position of A?

Credibility Feedback: Did U find the facts, logic, sources, and quotesin A to be truthful and credible or not?

Informon Qualities: How does the user rate the informons qualities, forexample, "interestingness," credibility, funniness, content value,writing quality, violence content, sexual content, profanity level,business importance, scientific merit, surprise/unexpectedness ofinformation content, artistic quality, dramatic appeal, entertainmentvalue, trendiness/importance to future directions, and opinionagreement.

Specific Reason Feedback: Why did the user like or dislike A?

Because of the authority?

Because of the source?

Because A is out-of-date (e.g. weather report from 3 weeks ago)?

Because the information contained in A has been seen already? (I.e., theproblem of duplicate information delivery)

Categorization Feedback: Did U liked A? Was it placed within the correctM and Z?

Such multi-faceted feedback queries can produce rich feedback profilesfrom U that can be used to adapt each of the profiles used in thefiltering process to some optimal operating point.

One embodiment of creating a MI profile (step 305) for each concept caninclude concept profiling, creation, and optimization. Broad descriptorscan be used to create a substantially-invariant concept profile, ideallywithout the word choice used to express concept C. A concept profile caninclude positive concept clues (PCC) and negative concept clues (NCC).The PCC and NCC can be combined by a processor to create ameasure-of-fit that can be compared to a predetermined threshold. If thecombined effect of the PCC and NCC exceeds the predetermined threshold,then informon A can be assumed to be related to concept C; otherwise itis eliminated from further processing. PCC is a set of words, phrases,and other features, such as the source or the author, each with anassociated weight, that tend to be in A which contains C. In contrast,NCC is a set of words, phrases, and other features, such as the sourceor the author, each with an associated weight that tend to make it moreunlikely that A is contained in C. For example, if the term "car" is inA, then it is likely to be about automobiles. However, if the phrase"bumper car" also is in A, then it is more likely that A related toamusement parks. Therefore, "bumper car" would fall into the profile ofnegative concept clues for the concept "automobile."

Typically, concept profile C can be created by one or more means. First,C can be explicitly created by user U. Second, C can be created by anelectronic thesaurus or similar device that can catalog and select froma set of concepts and the words that can be associated with thatconcept. Third, C can be created by using co-occurrence information thatcan be generated by analyzing the content of an informon. This meansuses the fact that related features of a concept tend to occur moreoften within the same document than in general. Fourth, C can be createdby the analysis of collections, H, of A that have been rated by one ormore U. Combinations of features that tend to occur repeatedly in H canbe grouped together as PCC for the analysis of a new concept. Also, an Athat one or more U have rated and determined not to be within aparticular Z can be used for the extraction of NCC.

Concept profiles can be optimized or learned continually after theircreation, with the objective that nearly all As that Us have foundinteresting, and belonging in M, should pass the predetermined thresholdof at least one C that can serve as an index into M. Another objectiveof concept profile management is that, for each A that does not fallinto any of the one or more M that are indexed by C, the breadth of C isadjusted to preserve the first objective, insofar as possible. Forexample, if C's threshold is exceeded for a given A, C's breadth can benarrowed by reducing PCC, increasing NCC, or both, or by increasing thethreshold for C.

In the next stage of filtering, one embodiment of content-based indexingtakes an A that has been processed into the set of C that describe it,and determine which M should accept the article for subsequentfiltering, for example, detailed indexing of incoming A. It is preferredthat a data structure including a database be used, so that the vectorof Ms, that are related to any concept C, may be looked-up. Furthermore,when a Z is created by U, the concept clues given by U to theinformation filter can be used to determine a set of likely concepts Cthat describe what U is seeking. For example, if U types in "basketball"as a likely word in the associated Z, then all concepts that have a highpositive weight for the word "basketball" are associated with the new Z.If no such concepts C seem to pre-exist, an entirely new concept C iscreated that is endowed with the clues U has given as the startingprofile.

To augment the effectiveness of concept-based indexing, it is preferredto provide continual optimization learning. In general, when a concept Cno longer uniquely triggers any documents that have been classified andliked by member clients U in a particular community M, then that M isremoved from the list of M indexed into by C. Also, when there appearsto be significant overlap between articles fitting concept C, andarticles that have been classified by users as belonging to M, and if Cdoes not currently index into M, then M can be added to the list of Mindexed into by C. The foregoing heuristic for expanding the concepts Cthat are covered by M, can potentially make M too broad and, thus,accept too many articles. Therefore, it further is preferred that areasonable but arbitrary limit is set on the conceptual size covered byM.

With regard to the detailed analysis of each informon A with respect tothe community profile for each M, each A must pass through this analysisfor each U subscribing to a particular M, i.e., for each member clientin a particular community. After A has passed that stage, it is thenfiltered at a more personal, member client level for each of thoseusers. The profile and filtering process are very similar for both thecommunity level and the member client level, except that at thecommunity level, the empirical data obtained is for all U who subscribedto M, and not merely an individual U. Other information about theindividual U can be used to help the filter, such as what U thinks ofwhat a particular author writes in other Zs that the user reads, andarticles that can't be used for the group-level M processing.

FIG. 6 illustrates the development of a profile, and its associatedpredictors, in accordance with the invention of this divisional patentapplication. Typically, regarding the structure of a profile 400, theinformation input into the structure can be divided into three broadcategories: (1) Structured Feature Information (SFI) 405; (2)Unstructured Feature Information (UFI) 410; and (3) Collaborative Input(CI) 415. Features derived from combinations of these three types act asadditional peer-level inputs for the next level of the rating predictionfunction, called (4) Correlated-Feature, Error-Correction Units (CFECU)420. From inputs 405, 410, 415, 420, learning functions 425a-d can beapplied to get two computed functions 426a-d, 428a-d of the inputs.These two functions are the Independent Rating Predictors (IRP) 426a-d,and the associated Uncertainty Predictors (UP) 428a-d. IRPs 426a-d canbe weighted by dividing them by their respective UPs 428a-d, so that themore certain an IRP 426a-d is, the higher its weight. Each weighted IRP429a-d is brought together with other IRPs 429a-d in a combinationfunction 427a-d. This combination function 427a-d can be from a simple,weighted, additive function to a far more complex neural networkfunction. The results from this are normalized by the total uncertaintyacross all UPs, from Certain=zero to Uncertain=infinity, and combinedusing the Certainty Weighting Function (CWF) 430. Once the CWF 430 hascombined the IRPs 426a-d, it is preferred that result 432 be shaped viaa monotonically increasing function, to map to the range anddistribution of the actual ratings. This function is called the CompleteRating Predictor (CRP) 432.

SFI 405 can include vectors of authors, sources, and other features ofinformon A that may be influential in determining the degree to which Afalls into the categories in a given M. UFI 410 can include vectors ofimportant words, phrases, and concepts that help to determine the degreeto which A falls into a given M. Vectors can exist for differentcanonical parts of A. For example, individual vectors may be providedfor subject/headings, content body, related information in otherreferenced informons, and the like. It is preferred that a positive andnegative vector exists for each canonical part.

CI 415 is received from other Us who already have seen A and have ratedit. The input used for CI 415 can include, for example,"interestingness," credibility, funniness, content value, writingquality, violence content, sexual content, profanity level, businessimportance, scientific merit, surprise/unexpectedness of informationcontent, artistic quality, dramatic appeal, entertainment value,trendiness/importance to future directions, and opinion agreement. EachCFECU 420 is a unit that can detect sets of specific featurecombinations which are exceptions in combination. For example, authorX's articles are generally disliked in the Z for woodworking, exceptwhen X writes about lathes. When an informon authored by X contains theconcept of "lathes," then the appropriate CFECU 420 is triggered tosignal that this is an exception, and accordingly a signal is sent tooffset the general negative signal otherwise triggered because of thegeneral dislike for X's informons in the woodworking Z.

As an example the form of Structured Feature Information (SFI) 405 caninclude fields such as Author, Source, Information-Type, and otherfields previously identified to be of particular value in the analysis.For simplicity, the exemplary SFI, below, accounts only for the Authorfield. For this example, assume three authors A, B, and C, havecollectively submitted 10 articles that have been read, and have beenrated as in TABLE 1 (following the text of this specification). In theaccompanying rating scheme, a rating can vary between 1 and 5, with 5indicating a "most interesting" article. If four new articles (11, 12,13, 14) arrive that have not yet been rated, and, in addition to authorsA, B, C, and a new author D has contributed, a simple IRP for the Authorfield, that just takes sums of the averages, would be as follows:

IRP(author)=weighted sum of

average(ratings given the author so far)

average(ratings given the author so far in this M)

average(ratings given all authors so far in this M)

average(ratings given all authors)

average(ratings given the author so far by a particular user U)*

average(ratings given the author so far in this M by a particular userU)*

average(ratings given all authors so far in this M by a particular userU)*

average(ratings given all authors by a particular user)*

* (if for a personal Z)

The purpose of the weighted sum is to make use of broader, more generalstatistics, when strong statistics for a particular user reading aninformon by a particular author, within a particular Z may not yet beavailable. When stronger statistics are available, the broader terms canbe eliminated by using smaller weights. This weighting scheme is similarto that used for creating CWFs 430, for the profiles as a whole. Some ofthe averages may be left out in the actual storage of the profile if,for example, an author's average rating for a particular M is not"significantly" different from the average for the author across all Ms.Here, "significance" is used is in a statistical sense, and frameworkssuch as the Minimum Description Length (MDL) Principle can be used todetermine when to store or use a more "local" component of the IRP. As asimple example, the following IRP employs only two of the above terms:

IRP(author)=weighted sum of

average (ratings given this author so far in this M)

average (ratings given all authors so far in this M)

Table 2 gives the values attained for the four new articles.

Uncertainty Predictors (UP) 428a-l can be handled according to theunderlying data distribution assumptions. It is generally important tothe uncertainty prediction that it should approach zero (0) as the IRPS426a-d become an exact prediction, and should approach infinity whenthere is no knowledge available to determine the value of an IRP. As anexample, the variance of the rating can be estimated as the UP. Asrecognized by a skilled artisan, combining the variances from thecomponents of the IRP can be done using several other methods as well,depending upon the theoretical assumptions used and the computationalefficiency desired. In the present example, shown in Table 3, theminimum of the variances of the components can be used. In thealternative, the UP 428a-l can be realized by: ##EQU1##

An example of Unstructured Feature Information (UFI) 410 can includeentities such as text body, video/image captions, song lyrics,subject/titles, reviews/annotations, and image/audio-extracted features,and the like. Using an exemplary entity of a text body, a sample of ten(10) articles that each have some number of 4 words, or tokens,contained therewithin are listed in TABLE 4. As before, a rating can befrom 1 to 5, with a rating of 5 indicating "most interesting." Thisvector can be any weighting scheme for tokens that allows for comparisonbetween a group of collected documents, or informons, and a document, orinformon, under question.

As previously mentioned, positive and negative vectors can provide aweighted average of the informons, according to their rating by user U.The weighting scheme can be based on empirical observations of thoseinformons that produce minimal error through an optimization process.Continuing in the example, weighting values for the positive can be:

    ______________________________________                                        Rating   5         4     3        2   1                                       ______________________________________                                        Weight   1.0       0.9   0.4      0.1 0.0                                     ______________________________________                                    

Similarly, the negative vector can use a weighting scheme in theopposite "direction":

    ______________________________________                                        Rating   5         4     3        2   1                                       ______________________________________                                        Weight   0.0       0.1   0.4      0.9 1.0                                     ______________________________________                                    

Using a TF-IDF scheme, the following token vectors can be obtained:

    ______________________________________                                                Token 1                                                                             Token 2     Token 3 Token 4                                     ______________________________________                                        Positive  0.71    0.56        0.33  0.0                                       Negative  0.30    0.43        0.60  0.83                                      ______________________________________                                    

In the case where four new documents come in to the information filter,the documents are then compared with the profile vector.

For the purposes of the example herein, only the TF-IDF representationand the cosine similarity metric, i.e., the normalized dot product, willbe used. TABLE 5 illustrates the occurrences of each exemplary token.TABLE 6 illustrates the corresponding similarity vector representationsusing a TF-IDF scheme. The similarity measure produces a result between0.0-1.0 that is preferred to be remapped to an IRP. This remappingfunction could be as simple as a linear regression, or a one-node neuralnet. Here, a simple linear transformation is used, where

    IRP(pos)=1+(SIM(pos))×4

and

    IRP(neg)=5-(SIM(pos))×4

TABLE 7 illustrates both IRP(pos) and IRP(neg), along with respectivepositive and negative squared-error, using the 14 articles, orinformons, read and rated thus far in the ongoing examples.

It is preferred that an estimate of the uncertainty resulting from apositive or negative IRP be made, and a complex neural net approachcould be used. However, a simpler method, useful for this example, issimply to repeat the same process that was used for the IRP but, insteadof predicting the rating, it is preferred to predict the squared-error,given the feature vector. The exact square-error values can be used asthe informon weights, instead of using a rating-weight lookup table. Amore optimal mapping function could also be computed, if indicated bythe application.

    ______________________________________                                                  Token 1                                                                             Token 2   Token 3 Token 4                                     ______________________________________                                        IRP pos. vector                                                                           16.68   8.73      12.89 11.27                                     IRP neg. vector                                                                           15.20   8.87       4.27  5.04                                     ______________________________________                                    

The UPs then can be computed in a manner similar to the IRP's:comparisons with the actual document vectors can be made to get asimilarity measure, and then a mapping function can be used to get anUP.

Making effective use of collaborative input (CI) from other users U is adifficult problem because of the following seven issues. First, theregenerally is no a priori knowledge regarding which users already willhave rated an informon A, before making a prediction for a user U, whohasn't yet read informon A. Therefore, a model for prediction must beoperational no matter which subset of the inputs happen to be available,if any, at a given time. Second, computational efficiency must bemaintained in light of a potentially very large set of users andinformons. Third, incremental updates of rating predictions often aredesired, as more feedback is reported from users regarding an informon.Fourth, in learning good models for making rating predictions, only verysparse data typically is available for each users rating of eachdocument. Thus, a large "missing data" problem must be dealt witheffectively.

Fifth, most potential solutions to the CI problem require independenceassumptions that, when grossly violated, give very poor results. As anexample of an independence assumption violation, assume that ten usersof a collaborative filtering system, called the "B-Team," always rateall articles exactly in the same way, for example, because they thinkvery much alike. Further assume that user A's ratings are correlatedwith the B-Team at the 0.5 level, and are correlated with user C at the0.9 level. Now, suppose user C reads an article and rates it a "5".Based on that C's rating, it is reasonable to predict that A's ratingalso might be a "5". Further, suppose that a member of the B-Team readsthe article, and rates it a "2". Existing collaborative filteringmethods are likely to predict that A's rating R_(A) would be:

    R.sub.A =(0.9×5+0.5×2)/(0.9+0.5)=3.93

In principle, if other members of the B-Team then read and rate thearticle, it should not affect the prediction of A's rating, R_(A),because it is known that other B-Team members always rate the articlewith the same value as the first member of the B-Team. However, theprediction for A by existing collaborative filtering schemes would tendto give 10 times the weight to the "2" rating, and would be:

    R.sub.A =(0.9×5+10×0.5×2)/(0.9+10×0.5)=2.46

Existing collaborative filtering schemes do not work well in this casebecause B-Team's ratings are not independent, and have a correlationamong one another of 1. The information filter according to the presentinvention can recognize and compensate for such inter-user correlation.

Sixth, information about the community of people is known, other thaneach user's ratings of informons. This information can include thepresent topics the users like, what authors the users like, etc. Thisinformation can make the system more effective when it is used forlearning stronger associations between community members. For example,because Users A and B in a particular community M have never yet readand rated an informon in common, no correlation between their likes anddislikes can be made, based on common ratings alone. However, users Aand B have both read and liked several informons authored by the sameauthor, X, although Users A and B each read a distinctly different Zs.Such information can be used to make the inference that there is apossible relationship between user A's interests and user B's interests.For the most part, existing collaborative filtering systems can not takeadvantage of this knowledge.

Seventh, information about the informon under consideration also isknown, in addition to the ratings given it so far. For example, fromknowing that informon A is about the concept of "gardening", better usecan be made of which users' ratings are more relevant in the context ofthe information in the informon. If user B's rating agrees with user D'srating of articles when the subject is about "politics", but B's ratingsagree more with user D when informon A is about "gardening", then therelationship between User B's ratings and User D's ratings are preferredto be emphasized to a greater extent than the relationship between UserB and User C when making predictions about informon A.

With regard to the aforementioned fourth, sixth and seventh issuesnamely, making effective use of sparse, but known, information about thecommunity and the informon, it is possible to determine the influence ofuser A's rating of an informon on the predicted rating of the informonfor a second user, B. For example, where user A and user B have read andrated in common a certain number of informons, the influence of user A'srating of informon D on the predicted rating of informon D for user Bcan be defined by a relationship that has two components. First, therecan be a common "mindset," S_(M), between user A and user B and informonD, that may be expressed as:

    M.sub.s =profile(A) X profile(B) X DocumentProfile(D).

Second, a correlation may be taken between user A's past ratings anduser B's past ratings with respect to informons that are similar to D.This correlation can be taken by weighting all informons E that A and Bhave rated in common by the similarity of E to D, S_(ED) :

    S.sub.ED =Weighted.sub.-- correlation(ratings(A),ratings(B))

Each of the examples can be weighted by ##EQU2## Note that the "X" inthe above equation may not be a mere multiplication or cross-product,but rather be a method for comparing the similarity between theprofiles. Next, the similarity of the member client profiles andinformon content profiles can be compared. A neural network could X beused to learn how to compare profiles so that the error in predictedratings is minimized. However, the invention can be embodied with use ofa simple cosine similarity metric, like that previously considered inconnection with Unstructured Feature Information (UFI) can be used.

The method used preferably includes more than just the tokens, such asthe author and other SFI; and, it is preferred that the three vectorsfor component also are able to be compared. SFIs may be handled bytransforming them into an entity that can be treated in a comparable wayto token frequencies that can be multiplied in the standard tokenfrequency comparison method, which would be recognized by a skilledartisan.

Continuing in the ongoing example, the Author field may be used. Whereuser A and user B have rated authors K and L, the token frequency vectormay appear as follows:

    ______________________________________                                              Avg.             Avg.         Avg.                                            Rating           Rating       Rating                                          Given to # in    Given to                                                                             # in  Given to                                                                             # in                               User  Author K sample  Author L                                                                             sample                                                                              Author M                                                                             sample                             ______________________________________                                        A     3.1      21      1.2    5     N/A    0                                  B     4         1      1.3    7     5      2                                  ______________________________________                                    

Further, the author component of the member client profiles of user Aand user B may be compared by taking a special weighted correlation ofeach author under comparison. In general, the weight is a function F ofthe sample sizes for user A's and user B's rating of the author, where Fis the product of a monotonically-increasing function of the sample sizefor each of user A and user B. Also, a simple function G of whether theinformon D is by the author or not is used. This function can be: G=q ifso, and G=p<q if not, where p and q are optimized constraints accordingto the domain of the filtering system. When there has been no rating ofan author by a user, then the function of the zero sample size ispositive. This is because the fact that the user did not read anythingby the author can signify a some indication that the author might notproduce an informon which would be highly rated by the user. In thiscase, the exact value is an increasing function H of the total articlesread by a particular user so far, because it becomes more likely thatthe user is intentionally avoiding reading informons by that author witheach subsequent article that has been read but is not prepared by theauthor. In general, the exact weighting function and parameters can beempirically derived rather than theoretically derived, and so is chosenby the optimization of the overall rating prediction functions.Continuing in the present example, a correlation can be computed withthe following weights for the authors K, L and M.

    ______________________________________                                        Author     Weight                                                             ______________________________________                                        K          F(21, 1, not author)                                                          = log(21 + 1) × log(1 + 1) × G(not author)                        = 0.04                                                             L          F(5, 7, author or D)                                                          = log(5 + 1) × log(7 + 1) × G(author)                             = 0.70                                                             M          F(0.2, not author)                                                            = H(26) × log(2 + 1) × G(not author)                              = 0.02                                                             ______________________________________                                    

It is preferred that the logarithm be used as themonotonically-increasing function and that p=1, q=0.1. Also used areH=log(sample₋₋ size*0.1) and an assumed rating, for those authors whoare unrated by a user, to the value of "2." The correlation for theauthor SFI can be mapped to a non-zero range, so that it can be includedin the cosine similarity metric. This mapping can be provided by asimple one-neuron neural network, or a linear function such as,(correlation+1)*P_(o). Where the P_(o) is an optimized parameter used toproduce the predicted ratings with the lowest error in the given domainfor filtering.

An artisan skilled in information retrieval would recognize that thereare numerous methods that can be used to effect informon comparisons,particularly document comparisons. One preferred method is to use aTF-IDF weighting technique in conjunction with the cosine similaritymetric. SFI including author, can be handled by including them asanother token in the vector. However, the token is preferred to beweighted by a factor that is empirically optimized rather than using aTF-IDF approach. Each component of the relationship between user A's anduser B's can be combined to produce the function to predict the ratingof informon D for user B. The combination function can be a simpleadditive function, a product function, or a complex function, including,for example, a neural network mapping function, depending uponcomputational efficiency constraints encountered in the application.Optimization of the combination function can be achieved by minimizingthe predicted rating error as an objective.

In addition to determining the relationship between two user's ratings,a relationship that can be used and combined across a large populationof users can be developed. This relationship is most susceptible to theaforementioned first, second, third, and fifth issues in the effectiveuse of collaborative input. Specifically, the difficulty with specifyinga user rating relationship across a large population of users iscompounded by the lack of a priori knowledge regarding a large volume ofdynamically changing information that may have unexpected correlationand therefore grossly violate independence assumptions.

In one embodiment of the present invention, it is preferred that usersbe broken into distributed groups called "mindpools." Mindpools can bepurely hierarchical, purely parallel, or a combination of both.Mindpools can be similar to the aforementioned "community" or mayinstead be one of many subcommunities. These multiple hierarchies can beused to represent different qualities of an article. Some qualities thatcan be maintained in separate hierarchies include: interestingness;credibility; funniness; valuableness; writing quality; violence content;sexual content; profanity level; business importance; scientific merit;artistic quality; dramatic appeal; entertainment value; surprise orunexpectedness of information content; trendiness or importance tofuture directions; and opinion agreement. Each of these qualities can beoptionally addressed by users with a rating feedback mechanism and,therefore, these qualities can be used to drive separate mindpoolhierarchies. Also, the qualities can be used in combinations, ifappropriate, to develop more complex composite informon qualities, andmore sublime mindpools.

FIG. 7 illustrates one embodiment of a mindpool hierarchy 500. It ispreferred that all users be members of the uppermost portion of thehierarchy, namely, the top mindpool 501. Mindpool 501 can be broken intosub-mindpools 502a-c, which separate users into those having at leastsome common interests. Furthermore, each sub-mindpool 502a-c can berespectively broken into sub-sub-mindpools 503a-b, 503c-d, 503e,f,g towhich users 504a-g are respective members. As used herein, mindpool 501is the parent node to sub-mindpools 502a-c, and sub-mindpools 502a-c arethe respective parent nodes to sub-sub-mindpools 503a-g. Sub-mindpools502a-c are the child nodes to mindpool 501 and sub-sub-mindpools 503a-gare child nodes to respective mindpools 503a-c. Sub-sub-mindpools 503a-gcan be considered to be end nodes. Users 505a,b can be members ofsub-mindpool 502a, 502c, if such more closely matches their intereststhan would membership in a sub-sub-mindpool 503a-g. In general, theobjective is to break down the entire population of users into subsetsthat are optimally similar. For example, the set of users who find thesame articles about "gardening" by author A to be interesting butnevertheless found other articles by author A on "gardening" to beuninteresting may be joined in one subset.

A processing means or mindpool manager may be used to handle themanagement of each of the mindpools 501, 502a-c, and 503a-g. A mindpoolmanager performs the following functions: (1) receiving ratinginformation from child-node mindpool managers and from those userscoupled directly to the manager; (2) passing rating information orcompiled statistics of the rating information up to the manager's parentnode, if such exists; (3) receiving estimations of the mindpoolconsensus on the rating for an informon from the manager's parentmindpool, if such exists; and (4) making estimations of the mindpoolconsensus on the rating for a specific informon for the users that comeunder the manager's domain; and (5) passing the estimations fromfunction 4 down to either a child-node mindpool or, if the manager is anend node in the hierarchy, to the respective user's CWF, for producingthe user's predicted rating. Function 4 also can include combining theestimations received from the manager's parent node, and UncertaintyPredictions can be estimated based on sample size, standard deviation,etc. Furthermore, as alluded to above, users can be allowed to belong tomore than one mindpool if they don't fit precisely into one mindpool buthave multiple views regarding the conceptual domain of the informon.Also, it is preferred that lateral communication be provided betweenpeer managers who have similar users beneath them to share estimationinformation. When a rating comes in from a user, it can be passed to theimmediate manager(s) node above that user. It is preferred that themanager(s) first decide whether the rating will effect its currentestimation or whether the statistics should be passed upward to aparent-node. If the manager estimation would change by an amount abovean empirically-derived minimum threshold, then the manager should passthat estimation down to all of its child-nodes. In the event that thecompiled statistics are changed by more than another minimum thresholdamount, then the compiled statistics should be passed to the manager'sparent-node, if any ,and the process recurses upward and downward in thehierarchy.

Because no mindpool manager is required to have accurate information,but just an estimation of the rating and an uncertainty level, anymanager may respond with a simple average of all previous documents, andwith a higher degree of uncertainty, if none of its child-nodes has anyrating information yet. The preferred distributed strategy tends toreduce the communication needed between processors, and the computationtends to be pooled, thereby eliminating a substantial degree ofredundancy. Using this distributed strategy, the estimations tend tosettle to the extent that the updating of other nodes, and the otherusers predictions are minimized. Therefore, as the number of informonsand users becomes large, the computation and prediction updates grow asthe sum of the number of informons and the number of users, rather thanthe product of the number of informons and the number of users. Inaddition, incremental updates can be accomplished by the passing ofestimations up and down the hierarchy. Incremental updates of ratingpredictions continue to move until the prediction becomes stable due tothe large sample size. The distributed division of users can reduce theeffects of independent assumption violations. In the previous examplewith the B-Team of ten users, the B-Team can be organized as aparticular mindpool. With the additional ratings from each of the B-Teammembers, the estimation from the B-Team mindpool typically does notchange significantly because of the exact correlation between themembers of that mindpool. This single estimation then can be combinedwith other estimations to achieve the desired result, regardless of howmany B-Team members have read the article at any given time.

The mindpool hierarchies can be created by either computer-guided orhuman-guided methods. If the hierarchy creation is human-guided, thereoften is a natural breakdown of people based on information such as jobposition, common interests, or any other information that is known aboutthem. Where the mindpool hierarchy is created automatically, thepreviously described measure of the collaborative input relationshipbetween users can be employed in a standard hierarchical clusteringalgorithm to produce each group of users or nodes in the mindpoolhierarchy. Such standard hierarchical clustering algorithms can include,for example, the agglomerative method, or the divide-and-conquer method.A skilled artisan would recognize that many other techniques also areavailable for incrementally-adjusting the clusters as new information iscollected. Typically, clustering is intended to (1) bring together userswhose rating information is clearly not independent; and (2) producemindpool estimations that are substantially independent among oneanother.

Estimations are made in a manner similar to other estimations describedherein. For example, for each user or sub-mindpool (sub-informant), asimilarity between the sub-informant and the centroid of the mindpoolcan be computed in order to determine how relevant the sub-informant isin computing the estimation. Uncertainty estimators also are associatedwith these sub-informants, so that they can be weighted with respect totheir reliability in providing the most accurate estimation. Optionally,the informon under evaluation can be used to modulate the relevancy of asub-informant. This type of evaluation also can take advantage of thetwo previously-determined collaborative information relationshipcomponents, thereby tending to magnify relationships that are strongerfor particular types of informons than for others. Once a suitable setof weights are established for each user within a mindpool for aparticular informon, a simple weighted-average can be used to make theestimation. It is preferred that the "simple" weighted average used bemore conservative regarding input information that a simple independentlinear regression. Also, the overall Uncertainty can be derived from theUncertainty Predictions of the sub-informants, in a manner similar tothe production of other uncertainty combination methods described above.Approximations can be made by pre-computing all terms that do not changesignificantly, based on the particular informon, or the subset of actualratings given so far to the mindpool manager.

As stated previously, the correlated-feature error-correction units(CFECUs) are intended to detect irregularities or statisticalexceptions. Indeed, two objectives of the CFECU units are to (1) findnon-linear exceptions to the general structure of the threeaforementioned types of inputs (SFI, UFI, and CI); and (2) findparticular combinations of informon sub-features that statisticallystand out as having special structure which is not captured by the restof the general model; and (3) trigger an additional signal to theCFECU's conditions are met, in order to reduce prediction error. Thefollowing exemplifies the CFECU operation:

    ______________________________________                                                      User B's Avg. Rating of                                                       of Informons About                                                            Gardening                                                                            Politics                                                 ______________________________________                                        Author A's      4.5      1.2                                                  Articles                                                                      Other Authors   1.4      2                                                    Weighted         1.68     1.87                                                by Topic                                                                      ______________________________________                                    

    ______________________________________                                                 User B's number of                                                            Informons Read About                                                                        Average over                                                    Gardening                                                                              Politics Topics                                             ______________________________________                                        Author A's  7          40      1.69                                           Articles                                                                      Other Authors                                                                            70         200      1.84                                           ______________________________________                                    

In this example, it is desired that author A's informon D aboutgardening have a high predicted rating for user B. However, because theaverage rating for author A by user B is only 1.69, and the averagerating for the gardening concept is only 1.68, a three-part model(SFI-UFI-CI) that does not evaluate the informon features in combinationwould tend to not rank informon D very highly. In this case, the firstCFECU would first find sources of error in past examples. This couldinclude using the three-part model against the known examples that userB has rated so far. In this example, seven articles that user B hasrated, have an average rating of 4.5, though even the three-part modelonly predicts a rating of about 1.68. When such a large error appears,and has statistical strength due to the number of examples with thecommon characteristics of, for example, the same author and topic, aCFECU is created to identify that this exception to the three-part modelhas been triggered and that a correction signal is needed. Second, it ispreferred to index the new CFECU into a database so that, whentriggering features appear in an informon, for example, author andtopic, the correction signal is sent into the appropriate CWF. Onemethod which can be used to effect the first step is a cascadecorrelation neural network, in which the neural net finds new connectionneural net units to progressively reduce the prediction error. Anothermethod is to search through each informon that has been rated but whosepredicted rating has a high error, and storing the informons profile.

When "enough" informons have been found with high error and commoncharacteristics, the common characteristics can be joined together as acandidate for a new CFECU. Next, the candidate can be tested on all thesamples, whether they have a high prediction or a low prediction errorassociated with them. Then, the overall error change (reduction orincrease) for all of the examples can be computed to determine if theCFECU should be added to the informon profile. If the estimated errorreduction is greater than a minimum threshold level, the CFECU can beadded to the profile. As successful CFECU are discovered for users'profiles, they also can be added to a database of CFECU's that may beuseful for analyzing other profiles. If a particular CFECU has asufficiently broad application, it can be moved up in the filteringprocess, so that it is computed for every entity once. Also, theparticular CFECU can be included in the representation that is computedin the pre-processing stage as a new feature. In general, the estimationof the predicted rating from a particular CFECU can be made by takingthe average of those informons for which the CFECU responds. Also, theUncertainty can be chosen such that the CFECU signal optimally outweighsthe other signals being sent to the CWF. One method of self-optimizationthat can be employed is, for example, the gradient descent method,although a skilled artisan would recognize that other appropriateoptimization methods may be used.

Many alterations and modifications may be made by those having ordinaryskill in the art without departing from the spirit and scope of theinvention. Therefore, it must be understood that the illustratedembodiments have been set forth only for the purposes of example, andthat it should not be taken as limiting the invention as defined by thefollowing claims. The following claims are, therefore, to be read toinclude not only the combination of elements which are literally setforth but all equivalent elements for performing substantially the samefunction in substantially the same way to obtain substantially the sameresult. The claims are thus to be understood to include what isspecifically illustrated and described above, what is conceptuallyequivalent, and also what incorporates the essential idea of theinvention.

                  TABLE 1                                                         ______________________________________                                        Article        Author  Rating given                                           ______________________________________                                         1             A       5                                                       2             B       1                                                       3             B       1                                                       4             B       5                                                       5             C       2                                                       6             C       2                                                       7             C       1                                                       8             C       2                                                       9             C       2                                                      10             C       2                                                      ______________________________________                                    

                                      TABLE 2                                     __________________________________________________________________________    Article Author normalized     normalized                                      IRP (author)                                                                          avg (author)                                                                         weight                                                                             weight                                                                            avg (all auth)                                                                      weight                                                                             weight                                     __________________________________________________________________________    11  A   5.00                                                                              3.12                                                                             0.86 2.40                                                                              0.49  0.14 4.65                                       12  B   2.67                                                                              0.23                                                                             0.32 2.40                                                                              0.49  0.66 2.49                                       13  C   1.83                                                                              6.00                                                                             0.92 2.40                                                                              0.49  0.06 1.86                                       14  D   N/A 0.00                                                                             0.00 2.40                                                                              0.49  1.00 2.40                                       __________________________________________________________________________

                  TABLE 3                                                         ______________________________________                                        Article                 var                                                   [alt.]                                                                              Author  var (author)                                                                            (all auth.)                                                                          UP (author)                                                                           UP (author)                            ______________________________________                                        11    A       2.25      2.04   2.04    1.07                                   12    B       4.33      2.04   2.04    1.39                                   13    C       0.17      2.04   0.17    0.15                                   14    D       N/A       2.04   2.04    2.04                                   ______________________________________                                    

                  TABLE 4                                                         ______________________________________                                        Article Token 1  Token 2  Token 3                                                                              Token 4                                                                              Rating                                ______________________________________                                        1       2        --       --     --     5                                     2       1        2        --     --     5                                     3       2        2        1      --     4                                     4       2        --       1      --     4                                     5       1        --       --     1      1                                     6       3        1        3      1      3                                     7       --       1        3      1      2                                     8       --       --       3      2      2                                     9       --       2        --     2      2                                     10      --       --       --     2      1                                     Frequency                                                                             11       8        11     9                                            ______________________________________                                    

                  TABLE 5                                                         ______________________________________                                        Article   Token 1 Token 2    Token 3                                                                             Token 4                                    ______________________________________                                        11        3       --         --    --                                         12        1       --         1     4                                          13        --      5          5     --                                         14        --      --         --    --                                         ______________________________________                                    

                  TABLE 6                                                         ______________________________________                                                                           Positive                                                                             Negative                            Article                                                                              Token 1 Token 2 Token 3                                                                             Token 4                                                                             Similarity                                                                           Similarity                          ______________________________________                                        1      0.18    0.00    0.00  0.00  0.73   0.26                                2      0.09    0.25    0.00  0.00  0.80   0.44                                3      0.18    0.25    0.09  0.00  0.96   0.60                                4      0.18    0.00    0.09  0.00  0.81   0.49                                5      0.09    0.00    0.00  0.11  0.54   0.73                                6      0.27    0.13    0.27  0.11  0.89   0.85                                7      0.00    0.13    0.27  0.11  0.55   0.89                                8      0.00    0.00    0.27  0.22  0.33   0.91                                9      0.00    0.25    0.00  0.22  0.50   0.76                                10     0.00    0.00    0.00  0.22  0.10   0.73                                11     0.27    0.00    0.00  0.00  0.73   0 26                                12     0.09    0.00    0.09  0.44  0.31   0.86                                13     0.00    0.63    0.45  0.00  0.67   0.64                                14     0.00    0.00    0.00  0.00  0.50   0.50                                ______________________________________                                    

                  TABLE 7                                                         ______________________________________                                        Article                                                                             IRP (pos)                                                                              IRP (neg)                                                                              Act. Rat.                                                                            sq. err. (pos)                                                                        sq. err. (neg)                         ______________________________________                                        1     3.93     3.95     5      1.14    1.09                                   2     4.19     3.25     5      0.66    3.06                                   3     4.84     2.61     4      0.71    1.94                                   4     4.23     3.03     4      0.05    0.94                                   5     3.18     2.09     1      4.74    1.18                                   6     4.58     1.61     3      2.50    1.93                                   7     3.21     1.44     2      1.45    0.32                                   8     2.31     1.37     2      0.10    0.40                                   9     3.01     1.96     2      1.03    0.00                                   10    1.41     2.09     1      0.17    1.20                                   11    3.93     3.95                                                           12    2.24     1.55                                                           13    3.68     2 44                                                           14    3.00     3.00                                                           ______________________________________                                    

What is claimed is:
 1. An information entity rating system comprising:acontent subsystem for receiving content-based profile data for aninformation entity and for combining content-based profile data for anindividual system user with the content-based profile data for theinformation entity to determine at least one computed rating functionindicating a content-based value of the information entity to the user;a collaboration subsystem for receiving collaborative input data for theinformation entity and for processing the collaborative input data todetermine at least one computed collaborative rating function indicatinga collaboration-based value of the information entity to the user; andan output subsystem for combining the content-based andcollaboration-based value functions to generate an output ratingpredictor of the informon for consideration by the user.
 2. The systemof claim 1 wherein:the content subsystem includes a structured featuresub-subsystem for receiving structured data of the content-based profiledata for the information entity and for combining structured data of thecontent-based profile data with the structured data of the informationentity to determine a computed structured-data rating functionindicating a structured content value of the information entity to theuser; the content subsystem further includes an unstructured featuresub-subsystem for receiving unstructured data of the content-basedprofile data for the information entity and for combining unstructureddata of the content-based profile data with the unstructured data of theinformation entity to determine a computed unstructured-data ratingfunction indicating an unstructured content value of the informationentity to the user; and the output system combines the structuredcontent-based, unstructured content-based, and collaboration-based valuefunctions in generating the output rating predictor.
 3. The system ofclaim 1 wherein:a correlation subsystem receives data from the contentsubsystem and from the collaboration subsystem to determine exceptionsto the computed rating functions on the basis of comparisons of dataincluded in the content-based and collaboration data and to generate anexception data value function indicating an opposing value to at leastone of the content-based and collaboration values; and the output systemfurther combines the exception data value function in generating theoutput rating predictor.
 4. The system of claim 2 wherein thecorrelation subsystem receives structured and unstructured data from thestructured feature and unstructured feature sub-subsystems anddetermines exceptions using data including the structured andunstructured data.
 5. The system of claim 3 wherein the contentsubsystem includes a structured feature sub-subsystem for receivingstructured data of the content-based profile data for the informationentity and for combining structured data of the content-based profiledata with the structured data of the information entity to determine acomputed structured-data rating function indicating a structured contentvalue of the information entity to the user;the content subsystemfurther includes an unstructured feature sub-subsystem for receivingunstructured data of the content-based profile data for the informationentity and for combining unstructured data of the content-based profiledata with the unstructured data of the information entity to determine acomputed unstructured-data rating function indicating an unstructuredcontent value of the information entity to the user, the correlationsubsystem further receives structured and unstructured data from thestructured feature and unstructured feature sub-subsystems anddetermines exceptions using data including the structured andunstructured data; and the output system combines the structuredcontent-based, unstructured content-based collaboration-based, andexception data value functions in generating the output ratingpredictor.
 6. The system of claim 1 wherein the content subsystem andthe collaboration subsystem employ respective learning functions incomputing value functions.
 7. The system of claim 1 wherein:the contentsubsystem determines at least one independent rating predictor and atleast one uncertainty predictor from which the content value function isdetermined; and the collaboration subsystem determines at least oneindependent rating predictor and at least one uncertainty predictor fromwhich the collaboration value function is determined.
 8. The system ofclaim 7 wherein each value function is determined by dividing theassociated independent rating predictor by the associated uncertaintypredictor.
 9. The system of claim 3 wherein:the content subsystemdetermines at least one independent rating predictor and at least oneuncertainty predictor from which the content value function isdetermined; the collaboration subsystem determines at least oneindependent rating predictor and at least one uncertainty predictor fromwhich the collaboration value function is determined; and thecorrelation subsystem determines at east one independent ratingpredictor and at least one uncertainty predictor from which theexception data value function is determined.
 10. The system of claim 1wherein the output system employs a certainty weighting function incombining the content and collaboration value functions.
 11. Aninformation processing system including the information entity ratingsystem of claim 1 wherein:a multi-level filter structure is providedwith a content-based filter containing content-based profile data whichincludes content-based data applicable to the individual user andcompares the filter content-based profile data with profile datarepresenting information in a network sourced informon; and thecontent-based filter determines whether the informon profile datasufficiently matches the user profile data of the content-based filter,and, if so, routing the informon to the information entity rating systemto obtain a rating of the informon for the individual user.
 12. Thesystem of claim 11 wherein:the content subsystem includes a structuredfeature sub-subsystem for receiving structured data of the contentprofile data for the information entity and for combining structureddata of the content-based profile data with the structured data of theinformation entity to determine a computed structured-data ratingfunction indicating a structured content value of the information entityto the user; the content subsystem further includes an unstructuredfeature sub-subsystem for receiving unstructured data of the contentprofile data for the information entity and for combining unstructureddata of the content-based profile data with the unstructured data of theinformation entity to determine a computed unstructured-data ratingfunction indicating an unstructured content value of the informationentity to the user; and the output system combines the structuredcontent-based, unstructured content-based, and collaboration-based valuefunctions in generating the output rating predictor.
 13. The system ofclaim 11 wherein:a correlation subsystem receives data from the contentsubsystem and from the collaboration subsystem to determine exceptionsto the computed rating functions on the basis of comparisons of dataincluded in the content-based and collaboration data and to generate anexception data value function indicating an opposing value to at leastone of the content-based and collaboration values; and the output systemfurther combines the exception data value function in generating theoutput rating predictor.
 14. An information entity rating systemcomprising:means for receiving content profile data for an informationentity and for combining content-based profile data for an individualsystem user with the content profile data for the information entity todetermine at least one computed rating function indicating acontent-based value of the information entity to the user; means forreceiving collaborative input data for the information entity and forprocessing the collaborative input data to determine at least onecomputed collaborative rating function indicating a collaboration-basedvalue of the information entity to the user; and means for combining thecontent-based and collaboration-based value functions to generate anoutput rating predictor of the informon for consideration by the user.15. The system of claim 14 wherein:the content data receiving meansincludes means for receiving structured data of the content-basedprofile data for the information entity and for combining structureddata of the content-based profile data with the structured data of theinformation entity to determine a computed structured-data ratingfunction indicating a structured content value of the information entityto the user; the content data receiving means further includes means forreceiving unstructured data of the content-based profile data for theinformation entity and for combining unstructured data of thecontent-based profile data with the unstructured data of the informationentity to determine a computed unstructured-data rating functionindicating an unstructured content value of the information entity tothe user; and the combining means combines the structured content-based,unstructured content-based, and collaboration-based value functions ingenerating the output rating predictor.
 16. The system of claim 14wherein:means are provided for receiving data from the content andcollaborative data receiving means to determine exceptions to thecomputed rating functions on the basis of comparisons of data includedin the content-based and collaborative data and to generate an exceptiondata value function indicating an opposing value to at least one of thecontent-based and collaboration values; and the combining means furthercombines the exception data value function in generating the outputrating predictor.
 17. A method for operating an information entityrating system, the method steps comprising:receiving content profiledata for an information entity and combining content-based profile datafor an individual system user with the content profile data for theinformation entity to determine at least one computed rating functionindicating a content-based value of the information entity to the user;receiving collaborative input data for the information entity andprocessing the collaborative input data to determine at least onecomputed collaborative rating function indicating a collaboration-basedvalue of the information entity to the user; and combining thecontent-based and collaboration-based value functions to generate anoutput rating predictor of the informon for consideration by the user.18. The method of claim 17 wherein:the content profile data receivingstep includes receiving structured data of the content-based profiledata for the information entity and combining structured data of thecontent-based profile data with the structured data of the informationentity to determine a computed structured-data rating functionindicating a structured content value of the information entity to theuser; the content profile data receiving step further includes receivingunstructured data of the content-based profile data for the informationentity and combining unstructured data of the content-based profile datawith the unstructured data of the information entity to determine acomputed unstructured-data rating function indicating an unstructuredcontent value of the information entity to the user; and the combiningstep combining the structured content-based unstructured content-based,and collaboration-based value functions in generating the output ratingpredictor.
 19. The method of claim 17 wherein the method steps furtherinclude:receiving correlated portions of the content profile data andthe collaborative input data to determine exceptions to the computedrating functions on the basis of comparisons of the correlated data andto generate an exception data value function indicating an opposingvalue to at least one of the content-based and collaboration values, andthe combining step further combines the exception data value function ingenerating the output rating predictor.
 20. The method of claim 17wherein:the content profile data receiving step includes determining atleast one independent rating predictor and at least one uncertaintypredictor from which the content value functionned; and thecollaborative input data receiving step includes determining at leastone independent rating predictor and at least one uncertainty predictorfrom which the collaboration value function is determined.
 21. A methodfor operating an information processing system including the informationentity rating method claim 17 wherein the method steps furtherinclude:operating a multi-level filter structure having a content-basedfilter containing content-based profile data which includescontent-based data applicable to the individual user; comparing thefilter content-based profile data with profile data representinginformation in a network sourced informon; and determining whether theinformon profile data sufficiently matches the user profile data of thecontent-based filter, and, if so, routing the informon to theinformation entity rating system to obtain a rating of the informon forthe individual user.